[rllib] Update flatten_observations.py for nested spaces for ignored multi-agent #59928
Conversation
…ent situations Signed-off-by: Mark Towers <[email protected]>
pseudo-rnd-thoughts
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request addresses a bug in the FlattenObservations connector for multi-agent scenarios with nested observation spaces. The core fix correctly preserves the original observation space for agents whose observations are not flattened, which is crucial for nested spaces. The changes are well-implemented and include several improvements: a new multi-agent example in the docstring that clearly demonstrates the fix, added type hints for better readability, and new assertions that enhance the robustness of the code by validating space types. The docstring for the __init__ method has also been updated for better clarity. Overall, this is a high-quality contribution that not only fixes the bug but also improves the code's maintainability and documentation.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Signed-off-by: Artur Niederfahrenhorst <[email protected]>
…multi-agent (ray-project#59928) ## Description For the FlattenObservation connector, in the mutli-agent case, if any of the agent's had a nested space, if it was ignored then we used the base_struct version of the observation_spaces rather than the actual spaces which for nested spaces is a problem. FYI, I believe that `get_base_struct_from_space` can be removed as Gymnasium has added the ability to treat `Dict` and `Tuple` spaces as like a dict or tuple for iterating, calling `.keys()`, `.values()` and `.items()`. For a future PR possibly ## Related issues Fixes ray-project#59849 --------- Signed-off-by: Mark Towers <[email protected]> Signed-off-by: Artur Niederfahrenhorst <[email protected]> Co-authored-by: Mark Towers <[email protected]> Co-authored-by: Artur Niederfahrenhorst <[email protected]> Co-authored-by: Artur Niederfahrenhorst <[email protected]> Signed-off-by: jasonwrwang <[email protected]>
2 participants
Description
For the FlattenObservation connector, in the mutli-agent case, if any of the agent's had a nested space, if it was ignored then we used the base_struct version of the observation_spaces rather than the actual spaces which for nested spaces is a problem.
FYI, I believe that
get_base_struct_from_spacecan be removed as Gymnasium has added the ability to treatDictandTuplespaces as like a dict or tuple for iterating, calling.keys(),.values()and.items(). For a future PR possiblyRelated issues
Fixes #59849